Modern “smart glasses” or augmented reality multisensory display devices (MSDs), such as Google Glass, may provide visual and auditory information to the wearer of the device beyond what they normally would see or hear without the device. MSDs may refer to any device that provides a heads-up display (e.g., a display in the current field of view) of visual information from an MSD or other computing device. MSDs may also provide the capability to recognize input speech. The speech recognition capability may be built-in, or may be enabled by sending and receiving recorded audio or speech recognition results to or from a network (e.g., the internet, the “cloud”) or another computing device. In various MSDs, the sensory information provided to the wearer of the device includes an augmentation of the environment around the wearer (visually or aurally displaying identification of objects around the wearer after object identification, such as a user's name when the face is identified). In various examples, the sensory information may be unrelated to the environment, such as providing an incoming text message visually or aurally. In various applications, combinations of sensory information augmenting the wearer's environment and sensory information unrelated to the environment are provided to the wearer.
Modern hearing assistance devices, such as hearing aids, typically include digital electronics to enhance the wearer's listening experience. Hearing aids are electronic instruments worn in or around the ear that compensate for hearing losses by specially amplifying sound. Hearing aids use transducer and electro-mechanical components that are connected via wires to the hearing aid circuitry.
Hearing aid users often find it difficult to understand speech in noisy conditions. Speech understanding in noisy conditions may be improved by providing the ability to replay audio from the last few seconds of speech. However, in such a case, the repeated speech might overlap with ongoing live speech. This problem with overlapping speech can be avoided by running a live speech recognizer on all speech around the hearing aid user (either all the time, or on-demand) and displaying the text on a screen such as that of a smartphone. However, this would require the user to look continuously at the phone's screen to read the recognition result.
What is needed in the art is an improved system for isolating the desired speaker's sound and displaying speech recognition output corresponding to the desired speaker's speech in noisy environments.